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Summary  Page 


PROBLEM: 

To  evaluate  the  accuracy  of  a  computer-assisted  dental  program  for  "classic"  presentations  of 
trauma  and  non- trauma  related  dental  pain. 


FINDINGS: 

The  diagnostic  accuracy  of  the  dental  program  for  "classic"  dental  cases  was  78%.  The 
program  correctly  distinguished  between  dental  conditions  requiring  immediate  evacuation  and 
those  which  could  reasonably  be  managed  aboard  submarines  by  independent  duty  corpsmen. 
Selection  of  the  appropriate  questionnaire  to  collect  data  was  critical  to  the  program’s  accuracy. 
The  program  was  sensitive  to  minor  variations  in  data  entry,  emphasizing  the  importance  of 
accurate  data. 


APPLICATION: 

Findings  can  assist  in  the  development  of  a  computer-assisted  diagnostic  program  for  dental 
pain  for  use  by  independent  duty  corpsmen  at  sea  and  serve  as  part  of  the  basis  for  a 
recommendation  for  service  use. 


ADMINISTRATIVE  INFORMATION 

This  investigation  was  conducted  under  Naval  Medical  Research  and  Development  Command 
Research  Work  Unit  MM33C30.002-5004.  It  was  submitted  for  review  on  March  1,  1989  and 
approved  for  publication  on  April  11,  1989.  It  has  been  designated  as  Naval  Submarine  Medical 
Research  Laboratory  Report  No.  1136. 
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ABSTRACT 


A  computer-assisted  dental  program  to  assist  independent  duty  corpsmen  in  the  diagnosis 
and  management  of  patients  who  present  at  sea  with  dental  pain  produced  the  correct  diagnosis 
78%  of  the  time  when  given  information  considered  by  dentists  to  be  "classic"  for  the  condition 
in  question.  The  program  correctly  distinguished  between  dental  conditions  which  require 
immediate  evacuation  and  those  which  can  be  managed  aboard  submarines  by  independent  duty 
corpsmen. 


Hi 


INTRODUCTION 


The  computer-assisted  program  for  dental  pain  was  first  developed  at  the  Naval  Dental 
Research  Institute,  Great  Lakes,  IL1,2  and  later  adapted  to  MS-DOS  format  and  implemented 
on  an  IBM-PC/AT  by  the  Naval  Submarine  Medical  Research  Laboratory,  Groton,  CT.  The 
program  is  rule  based,  designed  for  use  with  trauma  and  non-trauma  related  dental  pain,  and 
considers  a  total  of  35  dental  conditions  in  its  evaluation  of  patients  with  dental  pain. 

Nine  questionnaires  were  developed  to  evaluate  real-time  entry  of  patient  data  into  the 
program.  One  questionnaire  is  used  for  trauma  related  conditions  and  the  remaining  eight 
questionnaires  are  for  non-trauma  related  dental  conditions.  Table  1  lists  the  eight  categories  of 
non-trauma  related  dental  conditions.  Based  on  an  initial  evaluation  of  the  patient,  the  user 
selects  the  questionnaire  most  appropriate  for  the  presenting  dental  problem.  Each  questionnaire 
consists  of  a  list  of  questions  and  branch  points.  The  questions  cover  subjective  and  objective 
findings,  and  the  branch  points  direct  the  examiner  to  gather  further  information  depending  on 
the  responses  to  previous  questions.  The  branch  points  reflect  the  logic  of  the  diagnostic 
program.  If  the  examiner  follows  the  directions  of  the  branches,  he  will  collect  only  the 
information  required  by  the  program  for  a  diagnosis. 

Table  1.  Eight  questionnaires  used  to  collect  information  for  the  diagnosis  of  non-trauma 
related  dental  pain. 


Tooth  Specific 
Teeth,  Generalized/Multiple 
Gingiva,  Specific  Area 
Gingiva,  Generalized  Area 
Oral  Mucosa,  Tooth  Associated 
Temporomandibular  Joint/Muscle 
Dental  Extraction  Site 
Tissue  Swelling 


After  interviewing  the  patient  and  recording  findings  on  the  dental  questionnaire,  the 
corpsman  enters  the  data  into  the  diagnostic  program.  Upon  completion  of  data  entry,  the 
computer-assisted  dental  program  evaluates  symptom  findings  against  a  set  of  diagnostic  rules 
and  makes  a  diagnosis  when,  and  if,  the  conditions  of  a  rule  are  met.  Sometimes  the  program 
cannot  reach  a  diagnosis  based  on  the  information  and  displays  a  statment  to  that  effect.  The 
program  provides  both  "possible"  and  "probable"  dental  diagnoses  and  treatment  information. 
The  35  dental  conditions  considered  by  the  program  are  listed  in  Table  2.  Note  that  many  of 
the  diagnoses  are  not  mutually  exclusive  and  may  be  present  simultaneously.  Some  of  the 
diagnoses  are  quite  evident  on  examination,  but  were  included  to  provide  the  hospital  corpsman 
with  confirmation  of  his  own  diagnosis  and  rapid  access  to  related  treatment  information. 
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The  dental  program  is  designed  for  use  by  independent  duty  corpsmen  aboard  submarines 
in  their  management  of  dental  pain  at  sea.  Before  the  program  can  be  implemented  in  an 
operational  setting,  however,  its  accuracy  needs  to  be  evaluated.  This  paper  reports  on  the 
results  of  two  studies  which  examine  the  accuracy  of  the  computer-assisted  program  with  regard 
to  "classic"  presentations  of  trauma  and  non-trauma  related  dental  pain. 


Table  2.  Thirty-five  dental  conditions  considered  by  the  computer-assisted  progam  for  the 
diagnosis  of  trauma  and  non-trauma  related  dental  pain. 


Abscess/infection/cellulitis 
Acute  apical  abscess 
Acute  apical  periodontitis 
Acute  herpetic  gingivostomatitis 
Acute  gingivitis 
Carious  lesion  (decay) 

Dentin  hypersensitivity 
Defective  restoration 

Displace/mobility  of  tooth,  favorable  prognosis 
Displace/mobility  of  tooth,  guarded  prognosis 
Endo/perio  combined  problem 
Enamel  fracture 
Food  impaction 

Fractured  crown,  small  pulp  exposure 
Fractured  crown,  large  pulp  exposure 
Fractured  crown,  pulp  not  exposed 
Fractured  alveolar  bone 


Fractured  mandible 
Fractured  maxilla 
Fractured  facial  bones 
Irreversible  pulpitis 
Internal  derangement  of  the  TMJ 
Localized  alveolar  osteitis 
Maxillary  sinusitis 
Myofascial  pain/muscle  spasms 
Necrotizing  ulcerative  gingivitis 
Neurologic  injury 
Osseous  sequestrum 
Occlusal  trauma 
Periodontal  abscess 
Periocoronitis/erupting  tooth 
Reversible  pulpitis 
Root  fracture 

Total  avulsion,  good  candidate  for  replant 
Total  avulsion,  poor  candidate  for  replant 


STUDY  ONE 


Method 


Nineteen  dentists  at  the  Naval  Submarine  Base  Branch  Dental  Clinic,  Groton,  CT 
participated  in  the  first  study.  Using  the  questionnaires,  each  dentist  provided  "classic" 
responses  expected  from  interviewing  and  examining  patients  with  several  of  the  35  dental 
conditions.  Diagnoses  were  assigned  to  dentists  so  that  each  dentist  provided  information  for 
three  or  four  different  conditions.  Altogether,  two  sets  of  "classic"  responses  were  gathered  for 
each  condition  except  for  Abscess/Infection/Cellulitis,  for  which  only  one  was  obtained. 
Dentists  provided  cases  only  for  conditions  in  their  specialty  and  the  two  scenarios  for  a  given 
illness  were  completed  by  different  dentists. 

The  symptoms  for  each  of  the  resulting  69  "classic"  cases  were  entered  independently  into 
the  computer-assisted  diagnosis  program  by  three  research  investigators.  After  the  data  for  a 
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case  have  been  entered,  the  program  evaluates  the  symptom  information  against  a  set  of 
diagnostic  rules  and,  if  the  conditions  of  a  rule  are  met,  makes  a  diagnosis.  The  program  also 
classifies  the  diagnosis  as  either  possible  or  probable.  Sometimes  the  program  cannot  reach  a 
diagnosis;  in  these  cases,  a  statement  to  that  effect  is  displayed.  The  program  may  arrive  at 
several  possible  diagnoses  for  a  given  set  of  symptoms.  Often  the  several  diagnoses  describe 
related  aspects  of  the  same  condition.  The  diagnoses  made  by  the  computer  for  each  of  the  69 
"classic"  cases  were  recorded.  The  computer  program  was  considered  to  be  accurate  if  one  of 
the  diagnoses  made  by  the  program  as  either  probable  or  possible  was  identical  to  the  diagnosis 
for  which  the  "classic"  findings  had  been  supplied. 


Results 


Table  3.  Dental  conditions  classified  according  to  the  program’s  accuracy  for  both,  one  or 
neither  "classic"  presentation. 

Dental  conditions  for  which  the  program  produced  a  correct  diagnosis  for  both  "classic" 
presentations. 


Acute  apical  abscess 
Acute  apical  periodontitis 
Acute  gingivitis 
Carious  lesion  (decay) 

Dentin  hypersensitivity 
Defective  restoration 

Displace/mobility  of  tooth,  favorable  prognosis 
Displace/mobility  of  tooth,  guarded  prognosis 
Enamel  fracture 

Fractured  crown,  pulp  not  exposed 
Fractured  alveolar  bone 


Fractured  mandible 
Fractured  maxilla 
Fractured  facial  bones 
Irreversible  pulpitis 
Internal  derangement  of  the  TMJ 
Localized  alveolar  osteitis 
Necrotizing  ulcerative  gingivitis 
Osseous  sequestrum 
Periodontal  abscess 
Periocoronitis/erupting  tooth 
Total  avulsion,  good  for  replant 
Total  avulsion,  poor  for  replant 


Dental  conditions  for  which  the  program  produced  a  correct  diagnosis  for  one  of  two  "classic" 
presentations. 


Acute  herpetic  gingivostomatitis 
Food  impaction 

Fractured  crown,  small  pulp  exposure 
Maxillary  sinusitis 


Neurologic  injury 
Osseous  sequestrum 
Reversible  pulpitis 
Root  fracture 


Dental  conditions  for  which  the  program  failed  to  produce  a  correct  diagnosis  for  either  of  the 
"classic"  presentations. 


Abscess/infection/cellulitis*  Myofascial  pain/muscle  spasms 

Endo/perio  combined  problem  Reversible  pulpitis 

*For  Abscess/infection/cellulitis,  only  one  "classic"  presentation  was  entered,  and  the  computer 
misdiagnosed  the  presentation. 
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The  diagnostic  program  for  trauma  and  non-trauma  related  dental  pain  arrived  at  the 
correct  diagnosis  for  54  of  the  69  "classic"  cases  (78%  accuracy).  The  program  accurately 
diagnosed  both  "classic"  case  scenarios  for  23  of  the  35  diagnoses  considered  by  the  program. 
Dental  conditions  for  which  the  computer  was  accurate  for  both,  one  or  neither  case 
presentation  are  presented  in  Table  3. 

The  program  listed  only  a  single  diagnosis  for  30  of  the  69  case  presentations  (43%).  Of 
these  30  diagnoses,  24  (80%)  were  correct.  It  displayed  two  diagnoses  for  16  cases  (23%),  of 
which  10  (63%)  were  correct.  And  it  displayed  three  to  seven  diagnoses  for  the  remaining  23 
cases  (33%),  among  which  the  correct  diagnosis  was  included  in  20  cases  (87%). 

The  program  misdiagnosed  15  "classic"  cases.  First,  the  program  was  unable  to  arrive  at  a 
diagnosis  for  case  presentations  of  Maxillary  sinusitis  and  Myofascial  pain/muscle  spasm. 

Second,  the  program  misdiagnosed  both  cases  of  Endodontic/periodontic  combined  problem, 
Myofascial  pain/muscle  spasm,  Fractured  crown-large  pulp  exposure,  and  the  sole  case  of 
Abscess/infection/cellulitis.  Last,  the  program  misdiagnosed  one  of  the  two  presentations  of 
Reversible  pulpitis,  Maxillary  sinusitis.  Acute  herpetic  gingivostomatitis,  Food  impaction, 
Occlusal  trauma.  Fractured  crown-large  pulp  exposure.  Root  fracture  and  Neurologic  injury. 

While  diagnostic  precision  of  any  computer-assisted  diagnostic  aid  is  desirable,  the 
therapeutic  accuracy  of  the  program  has  greater  functional  importance  to  independent  duty 
corpsmen  at  sea.  With  this  in  mind,  each  dental  condition  was  classified  according  to  one  of 
three  kinds  of  therapeutic  interventions  available  at  sea.  These  are:  evacuate  immediately;  treat 
aboard  -  potential  for  condition  to  warrant  evacuation;  and  treat  aboard  -  return  to  duty. 

Dental  conditions  such  as  fractured  mandible,  fractured  maxilla,  and  fractured  facial  bones 
require  immediate  evacuation.  The  other  dental  conditions  are  treated  aboard  and  the  individual 
returned  to  duty,  or  treated  aboard  with  the  potential  for  evacuation.  The  program  was 
completely  reliable  in  that  it  never  suggested  immediate  evacuation  for  conditions  where  the 
usual  treatment  is  returning  to  duty.  In  addition,  the  program  never  suggested  returning  to  duty 
for  conditions  requiring  immediate  evacuation. 


Discussion 

The  program  produced  an  accurate  diagnosis  for  78%  of  these  "classic"  trauma  and  non¬ 
trauma  related  dental  cases.  When  accuracy  was  examined  according  to  treatment  rather  than 
diagnostic  precision,  the  program  reliably  distinguished  between  dental  conditions  requiring 
immediate  evacuation  and  those  for  which  treatment  is  returning  to  duty. 

The  program  made  more  than  one  diagnosis  for  most  of  the  cases.  In  general  the  other 
diagnoses  which  accompanied  the  "correct"  diagnosis  were  closely  related,  often  causal,  to  the 
dental  condition  for  which  a  dentist  had  supplied  the  findings.  Since  the  program  should  be 
most  accurate  for  "classic"  case  presentations,  diagnostic  accuracy  might  be  less  for  real  cases 
where  the  symptoms  are  not  so  clear  cut.  In  addition,  the  diagnostic  program  completely 
misdiagnosed  15  case  presentations.  An  examination  of  the  diagnostic  rules  for  each  of  these 
cases  showed  that  three  cases  were  misdiagnosed  because  the  wrong  questionnaire  was  selected 
to  record  the  symptoms.  For  example,  in  order  to  arrive  at  a  diagnosis  of  Myofascial 
pain/muscle  spasm,  the  user  must  complete  the  questionaire  for  dental  injuries  to  the 
Temporomandibular  Joint/Muscle;  instead,  the  questionnaire  for  Tooth  Specific  dental  injuries 
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was  used,  precluding  the  correct  diagnosis.  On  the  remaining  12  misdiagnosed  cases,  a  change 
in  the  response  to  one  question  for  11  of  the  cases  and  to  two  questions  for  the  other  case 
would  have  resulted  in  a  correct  diagnosis.  These  findings  indicate  how  unforgiving  the 
program  can  be  to  small  variations  in  symptomology  and  failing  to  use  the  appropriate 
questionnaire  to  interview  the  patient. 


STUDY  TWO 


The  first  study  left  a  number  of  unanswered  questions.  First,  were  the  findings  of  the  case 
truly  "classic"  to  the  particular  dental  diagnoses?  Second,  in  the  case  of  multiple  diagnoses, 
were  the  diagnoses  reasonable  in  consideration  of  the  findings  for  the  case?  And  third,  should 
the  program  have  considered  any  other  diagnoses  based  on  the  findings  provided  for  the  case? 
To  answer  these  questions,  a  second  study  was  conducted. 


Method 

The  findings  required  by  the  computer  to  make  a  diagnosis  for  each  of  the  case 
presentations  in  the  first  study  were  recorded  on  dental  questionnaires.  These  findings  were  not 
always  the  same  as  the  ones  provided  by  the  dentist  as  "classic"  to  a  diagnosis.  However,  when 
the  branching  of  the  questionnaire  was  carefully  followed,  the  "classic"  findings  provided  by  the 
dentist  and  those  required  by  the  computer  program  were  the  same. 

Questionnaires  were  distributed  to  the  19  dentists  who  participated  in  the  first  study.  Each 
dentist  was  asked  to  review  the  findings  for  three  or  four  cases.  No  dentist  reviewed  a  case  for 
which  he  had  originally  supplied  the  findings.  The  dentists  were  also  given  the  list  of  diagnoses 
generated  by  the  computer  for  each  case  they  reviewed  as  well  as  the  expected  diagnosis.  After 
reviewing  the  case,  the  dentist  responded  to  three  questions.  First,  were  each  of  the  diagnoses 
made  by  the  computer  for  the  case  reasonable  in  consideration  of  the  patient  information 
previously  provided  by  another  dentist?  Second,  were  the  findings  of  the  case  "classic"  to  the 
particular  diagnosis.  And  third,  were  there  other  diagnoses  the  computer  should  have 
considered,  but  failed  to  consider? 


Results 

On  40  of  the  69  cases  (72%),  the  dentists  reported  that  all  of  the  diagnoses  made  by  the 
computer  program  for  the  case  were  reasonable  in  consideration  of  the  findings  provided.  In 
addition,  the  dentists  considered  the  findings  to  be  "classic"  for  46  of  the  69  cases  (67%).  It  is 
interesting  that  of  the  15  cases  misdiagnosed  by  the  computer,  the  dentists  considered  only  two 
(13%)  to  be  "classic"  presentations  of  the  disease,  but  for  the  54  cases  correctly  diagnosed  by  the 
computer,  the  dentists  considered  44  of  the  cases  (81%)  to  be  "classic".  And  while  the  dentists 
considered  most  of  the  cases  to  be  "classic",  they  also  felt  that  the  computer  program  should 
have  considered  additional  diagnoses  for  a  little  more  than  a  third  of  the  cases  (36%). 
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Discussion 


The  second  study  helped  to  answer  a  number  of  questions  posed  by  the  results  of  the 
original  study.  The  computer-assisted  dental  program  arrived  at  multiple  diagnoses  for  many  of 
the  cases,  even  though  the  findings  provided  were  supposed  to  be  "classic"  to  a  particular 
diagnosis.  The  subsequent  study  showed  that  for  most  of  the  cases  (72%),  the  dentists  found  the 
alternate  diagnoses  reasonable  in  consideration  of  the  findings.  This  probably  is  because  the 
program  exhaustively  lists  all  diagnoses  which  are  suggested  by  the  findings,  which  tends  to 
include  related  and  often  causal  conditions  underlying  the  single  "classic"  diagnosis  for  which 
findings  were  provided.  In  fact,  the  dentists  occasionally  felt  that  the  program  failed  to 
consider  all  the  possible  diagnoses.  It  is  interesting  that  the  dentists  found  the  findings  "classic" 
for  81%  of  the  cases  accurately  diagnosed  by  the  dental  program,  but  only  13%  of  the 
misdiagnosed  cases.  Perhaps  it  is  not  as  easy  as  first  thought  to  identify  "classic"  findings  for 
certain  dental  conditions  and,  even  though  the  findings  for  a  case  may  be  "classic",  it  is  not 
possible  to  exclude  all  but  a  single  diagnosis. 


Summary 


These  two  studies  evaluated  the  accuracy  of  the  computer-assisted  dental  program  for 
"classic"  presentations  of  trauma  and  non-trauma  related  dental  pain.  The  overall  diagnostic 
accuracy  of  the  program  was  found  to  be  78%.  More  importantly,  the  program  correctly 
distinguished  between  dental  conditions  requiring  immediate  evacuation  and  those  for  which 
treatment  is  provided  aboard  and  the  individual  is  returned  to  duty.  The  program  was  not  as 
accurate  for  cases  where  findings  were  not  clearly  "classic".  While  the  program  arrived  at 
multiple  diagnoses  for  many  of  the  cases,  the  dentists  reported,  most  of  the  time,  that  the 
alternate  diagnoses  were  reasonable  in  consideration  of  the  findings.  Three  cases  were 
misdiagnosed  as  a  direct  result  of  the  wrong  questionnaire  being  selected  to  record  information, 
demonstrating  that  use  of  the  appropriate  questionnaire  is  critical  to  the  program’s  accuracy  and 
suggesting  a  need  for  the  program  to  guide  the  user  in  the  selection  of  a  questionnaire.  Data 
collected  in  an  initial  evaluation  of  the  patient  (e.g.  location  and  type  of  pain)  could  be  used  by 
the  program  to  select  the  appropriate  questionnaire  for  conducting  the  patient  interview.  Minor 
variations  in  responding  to  the  program’s  questions  had  significant  effects  on  the  diagnoses 
made  by  the  program.  This  finding  emphasizes  the  importance  of  good  data  collection  to 
program  accuracy.  Before  the  dental  program  can  be  used  by  Navy  hospital  corpsman  in  an 
operational  setting,  it  is  important  to  show  that  corpsmen  possess  the  ability  to  select  the 
appropriate  dental  questionnaire  to  use  in  interviewing  dental  patients  and  the  skills  necessary  to 
collect  accurate  data. 
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